Optical interconnector and highly interconnected, learning neural network incorporating optical interconnector therein

ABSTRACT

A variable weight optical interconnector is disclosed to include a projecting device and an interconnection weighting device remote from the projecting device. The projecting device projects a distribution of interconnecting light beams when illuminated by a spatially-modulated light pattern. The weighting device includes a photosensitive screen provided in optical alignment with the projecting device to independently control the intensity of each projected interconnecting beam to thereby assign an interconnection weight to each such beam. Further in accordance with the present invention, a highly-interconnected optical neural network having learning capability is disclosed as including a spatial light modulator, a detecting device, an interconnector according to the present invention, and a device responsive to detection signals generated by the detecting device to modify the interconnection weights assigned by the photosensitive screen of the interconnector.

BACKGROUND OF THE INVENTION

The present invention pertains to an optical interconnector and anoptical learning system or network incorporating the interconnectortherein. More particularly, the present invention pertains to an opticalinterconnector including an interconnection beam projection means whichprojects a distribution of light beams to interconnect a detecting meanswith an input means, and a weighting means, located in an optical pathbetween the projection means and either the input or the detectingmeans, which controls a parameter of each of the interconnecting beamsto set the interconnection strength or weight thereof. The opticallearning system according to the invention incorporates theinterconnector to provide a highly interconnected neural network havinglearning capability.

The present invention relates to an optical interconnector and anoptical learning system incorporating such an interconnector to providewhat has become known in the art as a "learning neural network". By wayof background, generally a neural network comprises devices thatsimulate the responses of biological neurons. A simplistic model for aneuron N₀ is shown in FIG. 1 to receive three inputs X₁, X₂, and X₃ at adevice which sums the inputs according to the simple equation S=X₁ +X₂+X₃. Positive X's may be defined as excitatory and tend to make themodel neuron "fire", that is provide a nonzero output. Negative X's,defined as inhibitory, tend to prevent the model neuron from firing. Anonlinear operator changes an output signal from the output S to thesumming device into a new signal according to a nonlinear, thresholdresponse curve. A low input signal to the nonlinear operator, that is asignal below some threshold, S₀, results in a zero output at thenonlinear operator. A high input signal gives a fixed axial output. Anintermediate input results in an intermediate output.

Output S' from the nonlinear operator is applied to still other neuronsafter multiplication by a weighting factor W by a distributor. Allsignals W₁ S', W₂ S', and W₃ S' are proportional to S'. The weightingfactor W controls the strength or weight of the connections betweenneuron N₀ and the summing elements of the three other neurons N₁, N₂,and N₃ shown in FIG. 1. If, for example, the weighting factor W₁ issmall, the interconnection between neurons N₀ and N₁ is said to be weakand thus the signal S₁ from neuron N₀ to neuron N₁ is attenuated.Conversely, if the weighting factor is large the signal S₁ is amplifiedand the interconnection between the neurons is said to be strong. It isseen that the information, memory, and problem solving methodscharacteristic of a neural network are determined by theinterconnections in network, that is what is interconnected to what andwith what strength.

Turning to the prior art, one recognizes U.S. Pat. No. 4,660,166 toHopfield as disclosing an earlier network which electronically simulatesneural activity to provide a system capable of retrieving particularinformation from a system memory in response to an interrogationthereof. The patentee describes such a retrieval system as anassociative memory, that is a memory that provides an output which isparticularly associated with a particular input applied to the system.In the Hopfield device an interconnected network of electronicamplifiers provides the "neurons".

To provide a neural network with capability to learn, theinterconnection strengths between the "neurons" in the network must bemodifiable. The neurons must be modifiable to provide a desired outputwhen the network is presented with an input. To provide modifiableneurons in an electrical network greatly complicates the overallelectronics. Moreover, electronic implementations appear to beinherently limited in the number of interconnections that can be madewithout regard to how difficult it is to change the strengths of theinterconnections. It is believed unlikely that an electronic circuitproviding for more that about 1,000,000 i.e. 1×10⁶ interconnections isfeasible as cross-talk problems and problems with power requirementsbecome overwhelming. In view of the shortcomings of electronicimplementations, optical learning networks have been developed. Such anetwork is described in an article by K. Wagner and S. Psaltis whichdiscloses a learning network utilizing a volume hologram comprisingphotorefractive crystals to interconnect nonlinear optical devices knownin the art as Fabry Perot etalons. The interconnections are both madeand weighted by the interference patterns in the photorefractivecrystal. Learning commences with the presentation of an optical input tothe network. The network initially will display an output which greatlydiffers from the desired output. To correct the variance between theactual and desired outputs, i.e. to induce the network to "learn", errorsignals are determined by taking the difference between the actual anddesired outputs. The error signals are sent backwards through thephotorefractive material as light rays to modify the interferencepatterns and thus the weight of the interconnections made by thehologram. Such modification is executed continuously until the generatedoutput matches or very nearly matches the desired output whereafter theerror signals are no longer permitted to propagate through the network.

The interconnection capacity of photorefractive material holograms alsois inherently limited. The density of the interference patterns in thephotorefractive material increases in proportion to the number ofinterconnections made by the hologram. As this density is increased, itbecomes increasingly difficult to modify the weight of someinterconnections without undesirably changing the strengths of otherinterconnections. These cross-talk problems in the interconnectioncapacity of photorefractive interconnecting holograms thereby inherentlylimit the number of modifiable interconnections which can be made in aneural learning network which incorporates such.

SUMMARY OF THE INVENTION

The optical interconnector according to the present invention overcomeslimitations inherent in prior art systems by providing a first elementdedicated to making optical interconnections and a separate, secondelement dedicated to assigning weights to the interconnections made bythe first element. The second element can be controlled to change theassigned interconnection weights or strengths during operation of theinterconnector to provide learning capacity in an optical neural networkincorporating such an interconnector. The active area of either thefirst or the second elements of the interconnector in accordance withthe present invention is not limited by crystal size or uniformity.Accordingly, the interconnector overcomes unavoidable limitations ininterconnection capacity in for example photoreactive crystal, volumehologram interconnectors.

The "first element" in the optical interconnector in accordance with thepresent invention comprises a projection means that projects adistribution of interconnecting light beams. In the preferredembodiments, the projecting means is provided by a holographic meanscomprising an array of spatially-localized holograms. The "secondelement", located remotely of the projecting means, amplitude modulateseach interconnection beam to encode the strength of each interconnectionmade by the beams. In the preferred embodiments, this "second element"comprises a photosensitive screen. The photosensitive screen controlsthe intensity of each interconnecting beam and may be completelytransparent, partially transparent or substantially opaque to anyparticular interconnecting beam. In the preferred embodiments, thescreen could comprise a photochromic, or a photodichromic material, aphotodichroic material, or a material having both photodichroic andphotochromic properties.

In a preferred embodiment of the interconnector where the photosensitivescreen comprises a photochromic material, the screen will change itsabsorption spectrum in response to exposure by light of a certainthreshold intensity. The screen is imaginarily divided into partitionsor cells in one-to-one correspondence with the holograms of theholographic means. Each cell or partition can be exposed to light havingthe requisite threshold intensity to locally change its absorptionspectrum to make such partition opaque, without affecting the opticaltransitivity of adjacent partitions. The screen partitions becomeprogressively less transmissive with continued exposure to light of thethreshold intensity. On the other hand, light having the samewavelength, but an intensity below the threshold can pass through thescreen without appreciably affecting the transitivity of the screen.

In an alternative embodiment of the interconnector, the photosensitivescreen is formed of a photodichroic material. Modification of thephotodichroic screen can be carried out by several known techniquesincluding direct writing or suppressive writing, and direct orsuppressive extinction writing.

In use, the interconnector in accordance with the present invention ispositioned to optically interconnect an input device, such as a spatiallight modulator and an output device such as a photodetector. Thepbotosensitive screen can be positioned in the optical path between theholographic means and the input device or between the holographic meansand the output device. When illuminated by an input spatial pattern froman input spatial modulator, the holographic means projectsinterconnection beams onto the output photodetector to fullyinterconnect the input and output devices. The projected interconnectingbeams are modulated according to the input spatial pattern. Theholographic means only switches the direction of the modulated lightfrom the input device and does not change the relative intensity of theswitched light. The photosensitive screen however does amplitudemodulate the interconnecting beams to change their relative intensitiesand thereby encode their interconnection strengths.

The disclosed optical, learning neural network which includes aninterconnection apparatus according to the present invention has aninterconnection capacity attaining 10¹ 2 interconnections. The preferredlearning network utilizes a page-oriented holographic means comprisingan N² ×N² array of spatially localized holograms arranged on a singleholographic plate or substrate. Each hologram projects oneinterconnecting beam so that the entire array provides for up to N⁴interconnections. The number N is contemplated to be very large and toequal or exceed 10³.

In the preferred network according to the present invention, atransmissive spatial light modulator (TSLM), defined to have N² pixels,modulates laser light in a particular way at each pixel thereof toprovide a modulated input or spatial pattern of N² light beams toilluminate the holographic means. The beam modulated by each pixel ofthe TSLM illuminates N² holograms of the holographic means. Each suchset of N² uniformly illuminated holograms divides the illuminating beaminto N² interconnecting beams and projects one interconnecting beam toeach of the corresponding N² pixels defined on the photodetector outputdevice to fully interconnect the input and output devices. Thephotosensitive screen is disposed in the optical path between either theinput TSLM and the holographic means or the holographic means and thephotodetector output to set the strengths of the interconnectionsbetween the input and output devices.

A modifying device controls the transitivity of the screen. In thepreferred embodiments, the modifying device comprises a computer and anoptical modifying means. The computer receives detection signals fromthe photodetector and generates an error signal based upon the detectionsignals, the input data injected into the network by the TSLM, andsignals representative of a desired output pattern. The error signalsare applied to the optical modifying means which amplifies theconstituent interconnecting beams in the output light pattern which areresponsible for generating the error signals. The amplified light isdirected back through the network onto the screen, to expose the screenand thereby change the absorption spectrum of selected cells. Thenetwork automatically, selectively reduces the transitivity of thosecells responsible for transmitting erroneous interconnecting beams inorder to modify the interconnection strengths and thereby reduce theerror signals as the generated output pattern more nearly resembles thedesired output for a given input.

A variable weight optical interconnector in accordance with the presentinvention comprises a projecting means and an interconnection weightingmeans remote from the projecting means. The projecting means projects adistribution of interconnecting light beams when illuminated by aspatial light pattern. The interconnection weighting means includes aphotosensitive screen provided in optical alignment with the projectionmeans. The screen has variable transitivity to control a parameter ofeach of the interconnecting light beams and thereby assign aninterconnection weight to each such beam.

A highly interconnected optical neural network having capability tolearn in accordance with the present invention comprises a spatialmodulating means responsive to an input signal for modulating light toprovide modulated optical signals, detecting means for providingdetection signals in response to light incident thereon, interconnectormeans for providing a distribution of interconnecting light beams to thedetecting means when illuminated by the modulated optical signals tointerconnect the modulating means and the detecting means, and meansresponsive to the detection signals for modifying the interconnectionweights assigned by the weighting means. The interconnector meansincludes projection means which projects interconnecting beams withoutchanging a parameter of the beams relative to each other, and aninterconnection weighting means provided in an optical path of theprojection means for controlling a parameter of each interconnectinglight beam relative to the other beams so as to assign aninterconnection weight to each beam and thereby provide weighted,modulated interconnecting beams.

BRIEF DESCRIPTION OF THE DRAWINGS

Further aspects and features of the present invention will be even moreapparent from the following detailed description and drawings, and theappended claims. In the drawings:

FIG. 1 is a schematic view illustrating a model for an interconnectedneuron;

FIG. 2 is a block diagram of a preferred embodiment of the neuralnetwork which incorporates the interconnector according to the, presentinvention;

FIG. 3 is a schematic plan view of a preferred interconnector accordingto the present invention, which interconnector comprises aphotosensitive screen in optical alignment with an N² ×N² array ofholograms provided in a page-oriented holographic means;

FIG. 4 illustrates a sigmoidal, monotonic input-output responsecharacteristic utilized in the neural network of FIG. 2;

FIG. 5 is a ray diagram useful in illustrating one possible opticalinterconnection of the input and detection elements in the network ofFIG. 2;

FIG. 6 is a ray diagram useful in illustrating modification of thetransitivity of the screen in the network of FIG. 2;

FIG. 7 is a block diagram showing an alternative embodiment of aninterconnector in accordance with the present invention; and

FIG. 8 is a block diagram of a composite neural network incorporatingseveral neural networks of the type shown in FIG. 2.

DETAILED DESCRIPTION OF THE INVENTION

In schematic form, FIG. 2 shows a preferred, highly interconnectedneural network 10 having learning capability in accordance with thepresent invention. It is contemplated that preferred network or system10 can be implemented completely with optical elements. However, as willbe seen, network 10 could comprise various combinations of optical andelectrical elements while remaining well within the scope of the presentinvention.

Network 10 is seen to include a laser 12 providing a source of laserlight at a particular wave-length. Laser light from laser source 12 isdirected onto a spatial light modulator (TSLM) 14. Spatial lightmodulator 14 is of the polarizing transmissive type, and so when laserlight from source 12 is transmitted through TSLM 14, it is modulated toprovide a spatial pattern according to input data programmed into theTSLM. Modulated light transmitted through TSLM 14 is expanded by a lenssystem 16 comprising a double concave lens 18 and a focusing lens 20.Lens system 16 directs the expanded, modulated beams from TSLM 14through an optical interconnector 22 in accordance with the presentinvention. Preferred interconnector 22 comprises a photosensitive,transmissive screen 24 and a page-oriented holographic means 26.Holographic means 26 is formed to project N. light beams to beamsplitter28 when the holographic means is illuminated. It will be seen that theintensity of each beam projected by holographic means 26 depends uponthe particular spatial distribution of modulated light transmitted byTSLM 14, which modulated light illuminates the holographic means, andthe transmissive condition of photosensitive screen 24. Beamsplitter 28splits the beams projected by holographic means 26 and directs the splitbeams onto designated pixels at the photodetecting surface 30 of adetector 32 and onto associated pixels defined on the reflective side 34of a reflective spatial light modulator (RSLM) 36. Detection signalsfrom the detector 32 are applied to computer 38. Computer 38 alsoreceives signals indicative of a desired output spatial pattern from adesired pattern memory 40 and signals representative of the input signalpattern which are applied simultaneously to TSLM 14. Based upon acomparison of the detection signals and the desired output patternsignals, for a given input signal pattern, computer 38 generates errorsignals and applies the error signals to RSLM 36. A phase conjugatemirror (PCM) 42 is disposed to receive light reflected from thereflective face 34 of RBLM 36. The PCM 42 amplifies and reversepropagates the received light back to RSLM 36 which then reflects theamplified light to expose transmissive screen 24.

Interconnector 22 provides for complete interconnection between eachpixel defined on TSLM 14 and each pixel defined on the photodetectionsurface 30 of detector 32. Interconnector 22 is depicted in plan view inFIG. 3 as comprising phototransmissive screen 24 in optical alignmentwith holographic means 26. Holographic means 26 is the element whichmakes the interconnections in network 10 and therefore will be discussedfirst. In the preferred embodiment, holographic means 26 is of thepage-oriented type which comprises N' individual holograms 44 arrangedin an N*×N*, spatially localized array. Holograms 44 are affixed in atransparent substrate such as a glass slide. As is well known, eachindividual hologram 44 can be formed from a thermoplastic material or aphotorefractive material. In the preferred system 10, holograms 44 arepermanently recorded. Alternatively, if desired, holograms 44 could beerasable. Those of ordinary skill in the art also appreciate thatspatially localized holograms 44 could be of the planar type or could bevolume holograms. Further still, it will be understood that theinterconnections between TSLM 14 and photodetection surface 30 could bemade by an array of lenses and/or mirrors However, holographic means 26is the preferred element to provide for 10¹ 2 or more interconnectionsin network 10.

The TSLM 14 is defined to have N² pixels Photodetecting surface 30 andreflecting surface 34 likewise are defined to have N² pixels. Each pixelof TSLM 14 modulates a portion of the light from laser 12 and transmitsthe modulated light therethrough to provide a spatial pattern formed ofN² modulated light beams. Holographic means 26 and TSLM 14 are opticallyaligned so that the modulated light beam from each pixel of TSLM 14illuminates N² holograms 44 of the holographic means. Each hologram 44in turn switches the direction of the modulated light from TSLM 14 toproject a modulated interconnecting beam through beamsplitter 26 to aparticular pixel on photodetection surface 30. By way of example, whenthe ith group of N² holograms 44 are illuminated by light from the ithpixel of TSLM 14, these N² holograms will direct one interconnectingbeam, modulated or encoded by the intensity of the light passed by theith pixel of TSLM 14, to each of the N² pixels of photodetection surface30 (and the reflecting face 34 of RSLM 36). It is thus understood thatholographic means 26 divides the N² modulated light beams from TSLM 14which form the input spatial pattern and switches their direction toproject N⁴ modulated interconnecting beams through beamsplitter 28. Allof the N⁴ holograms 44 of holographic means 26 have the sametransitivity so that the relative intensities of the projectedinterconnecting beams will not be changed by the holographic means.Indeed, if holographic means 26 were directly exposed to light fromlaser 12, without the affects of TSLM 14 and transmissive screen 24,each interconnecting beam projected by the holographic means would havethe same intensity and therefore the same interconnection weight orstrength.

Phototransmissive screen 24 assigns the weight or strength to eachinterconnection made by interconnector 22. As illustrated in FIG. 3,screen 24 is divided into imaginary partitions or cells 46 delineated byalternating dot-dash lines. There is no physical boundary between thepartitions 46. Screen 24 and holographic means 26 are aligned such thatthere is a one-to one correspondence between each partition 46 and eachhologram 44 and therefore each interconnection beam projected by theholographic means. Accordingly, screen 24 is defined to have N⁴ suchpartitions 46 likewise arranged in an N² ×N² array. The intensity ofeach interconnection beam is controlled by the optical transitivity ofeach partition 46 to assign an interconnection strength to each suchbeam.

Preferred screen 24 comprises a sheet of photochromogenic material suchas a photochromic or photodichromic glass. In the preferred embodiments,screen 24 also could comprise a photodichroic material. It is alsocontemplated that screen 22 may comprise photographic emulsions.Regardless of which type of material is selected to form screen 24, thescreen is fabricated so that when one or more of its cells or partitions46 are exposed to light having a certain parameter, the screen 22 reactsby modifying its absorption spectrum in the exposed partition orpartitions. Where photosensitive screen 24 is constructed from aphotochromic material, a photochromic glass such as Corning fast fadeNo. 8111 glass or a Corning experimental glass known as UXC arepreferred. The photodichromic glass darkens where it is exposed to lighthaving an intensity which exceeds a characteristic threshold intensityfor the glass. Hereinafter, this characteristic intensity will bereferred to as the modification threshold intensity, that is the beamintensity necessary to modify the optical transitivity of the glass. Thedegree of darkening depends upon the actual intensity of the exposinglight and the duration of exposure. If the exposure light intensityfails to exceed the threshold modification intensity, the light can betransmitted through the screen without appreciably affecting thetransitivity of the exposed partitions 46 thereof.

It follows that where interconnector 22 comprises a photochromic screen24 in preferred system 10, laser 12 is provided to have an carrier beamwith an intensity somewhat below the threshold intensity of the screen.The TSLM 14 modulates in any of several know ways without amplifying theintensity of the light. The PCM 42 and the RSLM 36 provide the opticalmeans for carrying out modification of the interconnection strengths byamplifying light transmitted through screen 24 to above the modificationthreshold and directing the amplified light to expose selectedpartitions of the screen. It is the province of the PCM 42 to amplifythe exposure light to the necessary intensity. The RSLM 36 directs theamplified light to the desired partitions 46 to be modified.

The phase conjugating materials from which PCM 42 is provided are nowwell known in the art. PCM 42 is a nonlinear optical device whichreverses the direction of propagation of probe beams incident on it soas to provide return beams which exactly retrace the path of the probebeams back to their points of origin. As explained in U.S. Pat. No.4,739,496, a diverging light beam emanating from a point source will,upon reflection from a phase conjugating device, result in a convergingbeam which propagates back to its origin point in space. This propertyis referred to in the art as wave front reversal or time-reversalreflection.

To suitably amplify the return beams, PCM 42 could be of the degeneratefour wave mixing type wherein two pump beams having the same frequencyare directed to set up a standing wave in a transparent material whoseindex of refraction varies with intensity. When the material is exposedby probe beams also having the same frequency, the material emitstime-reversed return beams which are amplified by the pump beams.Numerous other phase conjugating devices are readily available 5 andcould be used in system 10. Such devices in include three-wave or otherfour-wave mixing photon scatter devices, Kerr-like phase conjugationdevices, stimulated scattering or back scattering devices, nonlinearphoton echo devices, surface phase conjugation devices, and varioussemiconductor or photorefractive devices.

Computer 38 determines how the interconnection strengths set byphotosensitive screen 24 are to be modified and controls thereflectivity of RSLM 36 accordingly. Computer 38 could be an electroniccomputer but it is also contemplated that computer 38 can be constructedfrom optical components. Computer 38 receives detection signals havinganalog values from detector 32 and signals representative of a desiredpattern from its memory 40. Mathematically, the detection signalsreceived from detector 30 can be thought of as a N² ×1 vector which willbe referred to as Y'. The desired pattern signals would be considered asrepresenting a N² ×1 vector T. Computer 38 generates error signals,which will be considered as another N² ×1 vector E, as a function of thevectors Y' and T. The particular function selected for generation of theerror signals depends upon the application of preferred system 10. Anyof several well known algorithms may be selected.

To generate the error signal vector E, computer 38 compares thedetection signals Y' and the desired pattern signals T and calculatesthe error signals in proportion to the differences between the detectionand the desired pattern signals. In detail, detector 32 generates adetection signal in response to the intensity of light at each pixel onits photodetecting surface 30. Detector 32 applies N² of these detectionsignals in parallel to computer 38 as vector Y'. Computer 38 compareseach component signal y₁ ' of Y' with a corresponding element t: of T togenerate a series of error signals which form the error vector E whichis applied to RSLM 36. Each pixel of photodetecting surface 28 has acorresponding pixel on RSLM 36. When the error vector E is applied toRSLM 36, the RSLM responds by altering the reflectivity of the definedpixels of its reflective surface 34. The reflectivity of the ith pixelof surface 34 is changed in proportion to the difference between theelement y_(i) ' of vector Y' and the element t_(i) of vector I. Ifcomputer 38 determines that y_(i) equals t_(i), the error signalrepresentative of e_(i), when applied to RSLM 36, will cause the ithpixel of RSLM 36 to be made substantially less reflective and preferablysubstantially nonreflective. On the other hand, a large differencebetween a signal y_(j) and t_(j) translates to a small reduction or noreduction in the reflectivity of the ith pixel of RSLM 36. Thus, RSLM 36only weakly partially reflects to PCM 42 each distribution of N*modulated interconnecting beams which provide a detection signal nearlyequal to the desired pattern signal when detected at detector 32. TheRSLM 36 fully reflects each group of modulated interconnecting beamswhich cause generation of detection signals that are substantiallydifferent from their corresponding desired pattern signals.

The pattern of partially and strongly reflected beams from RSLM 36 arethe probe beams for PCM 42. The PCM 42 generates return beams havingamplified intensities proportional to the probe beams from RSLM 36. TheRSLM 36 in turn reflects the amplified return beams from PCM 42 in thesame manner that it directs the modulated interconnecting beams as probebeams onto PCM 42 so that the relative low intensity return beams whichcorrespond to the weakly reflected probe beams are themselves onlypartially reflected and the relatively high intensity return beams whichcorrespond to the substantially fully reflected probe beams likewise arefully reflected by the RSLM. All of the return beams are directedthrough holographic means 26 to illuminate photosensitive screen 24. Dueto the amplification of the controlled reflectivity of the pixels ofreflecting surface 32, the fully or nearly fully reflected return beamshave intensities exceeding the modification threshold for screen 24. Thehighly reflected return beams thus darken their corresponding screenpartitions 46. The partitions 46 darkened are those that first pass andencode the interconnecting beams which first become the highly reflectedprobe beams and thereafter, the high intensity return beams. Theinterconnection strengths set by the darkened partitions 44 are thusreduced to correspondingly reduce the error signal components e_(i) withwhich they are associated. The weaker the reflected return beam directedonto screen 24 by RSLM 36, the less effect the return beam will have onthe transitivity of its associated screen partition 46. The weakestreflected return beams will have intensities less than the screenthreshold modification intensity and thus leave their associatedpartitions unchanged.

To generate the N² signals defining vector Y', detector 32 performs adetection function and a nonlinear operation. The detection function iselementary and will be considered first. Each pixel of photodetectingsurface 30 is exposed to N² interconnecting beams projected byholographic means 26. Detector 32 sums the intensities of the N² beamsincident on each of its N² pixels to obtain a summed intensity value ateach pixel. With reference to the mathematical representationsintroduced supra., the N² summed intensity values provide the elementsy_(i) of a 1×N² column vector Y.

Detector means 32 then performs a nonlinear transformation o& each ofthe N² detected intensity values y_(i) to obtain a new value y_(i) '.FIG. 4 is a graphical representation of a function G(Y) which describesthe transformation carried out by detector 32 for each summed intensityvalue y_(i). The function G(Y) is a continuous, monotone increasing,nonlinear function which maps the vector Y in N² space to the new vectorY' in N² space to provide a sigmoidal threshold response. Detector 32can be controlled to set the particular slope or steepness of itssigmoidal response curve G(Y) depending upon different applications ofsystem 10. Preferably detector 32 performs the nonlinear transformationof each of the summed intensity values in parallel to apply the vectorY' in parallel to computer 38.

In describing the operation of preferred system 10, reference will bemade to a series of mathematical operations that represent a series ofprocessing steps which the system automatically carries out. Accordingto a mathematical description, system 10 is understood as performingfive distinct steps. The first step is a matrix vector multiplicationgiven by

    AX=Y

where X is the input vector that represents an input pattern which, forexample, system 10 may be made to learn for subsequent identification ofsuch pattern. As described above input vector X may be thought of as aN² ×1 column vector in the form ##EQU1## Input vector X is programmedinto system 10 by means of TSLM 14. The TSLM 14 in turn imparts to thelight from laser 12, an input spatial pattern representative of theinput vector X. As TSLM 14 is defined to have N² pixels, each pixeltransmits light representing one element x_(i) of X.

Holographic means 26 and photosensitive screen 24 insert the matrix Ainto system 10. Corresponding to the N² ×1 vector X, the matrix A is anN² ×N² matrix having N elements The magnitude of each element a_(i) j ofA is encoded by photosensitive screen 24. The magnitude of the elementsa_(i) j, represent the strength of each interconnection made betweenTSLM 14 and detector 32. Light from each pixel of TSLM 14 is switchedinto N² separate beams by holographic means 26. Holographic means 26thus projects N² separate beams corresponding to each pixel of TSLM 14onto an associated pixel on detecting surface 30 and an associated pixelon reflecting surface 34. When photodetecting surface 30 sums theintensities of the N² interconnecting beams incident on each pixelthereof, the matrix multiplication

    AX=Y

is completed.

The second step, written as

    NL(Y)=Y',

is carried out by detector 32. Detector 32 nonlinearly transforms theintensity value registered at each detection pixel according to thesigmoid g(y_(i))=y_(i) ' graphically illustrated in FIG. 4. Theresulting vector Y' is applied to computer 38 for comparison andcalculation of the error signal vector E.

Computer 38 determines the vector E according to a function of Y' and T.This step is written as

    E=F(X, Y', T).

The particular function, F, selected for generation of vector E can beany of several well known learning algorithms and could simply bec(Y'-T)^(1/2) where c is the amplification factor of PCM 42. The vectorE is also written as a function of X because each particular desiredpattern T is associated with a particular input pattern X.

Computer 38 applies vector E to RSLM 36 to set the reflectivity of eachpixel of RSLM 36 accordingly. The weighted interconnecting beamsmodulated by TSLM 14 are reflected by RSLM 36 as probe beams onto PCM42. The intensities of the N⁴ probe beams are controlled by thereflectivity of each pixel defined on reflective face 34. Thereafter,PCM 42 time reverse reflects and amplifies the probe beams to provideamplified return beams to RSLM 36. Pixels of RSLM 36 controlled to bepoorly reflective and pixels adjusted to be highly reflective furtheradjust the intensities of the return beams and then direct them toilluminate phototransmissive screen 24. This double reflection of theoptical signals representing vector Y as probe beams from RSLM 36 ontoPCM 42 and back as amplified return beams from the PCM completes anoperation described by the vector multiplication

    Z=E.sup.2 Y.

The light beams indicative of the vector Z are amplified by thecharacteristic gain of PCM 42 to have sufficient intensity to alter thetransitivity of selected partitions 46 of photosensitive screen 24 andthereby change the interconnection strengths according to the relation

    Δa.sub.i j =ce.sup.2.sub.j X.sub.i a.sub.i j

where c is the amplification of the PCM 42. In the initial state ofoperation of system 10, there would be no error signal and thus noconjugate beams from PCM 42. Then, a new spatial pattern is transmittedto TSLM 14 to begin a new cycle of learning.

In connection with the operation of the preferred neural networkaccording to the invention, FIGS. 5 and 6 are partial ray diagramsuseful in illustrating the optical paths between the elements in thenetwork. The example in FIGS. 5 and 6 of course is greatly simplifiedwhere, for purposes illustration, N is considered to equal 2. It isunderstood by those of ordinary skill in the art that FIGS. 5 and 6 areonly one example of the optical paths between the elements shown andthat various alternative paths are apparent. As holograms 44 do notaffect the modification of the transitivity of screen 24, the hologramsare omitted in FIG. 6.

With N considered to equal 2 and with reference to both FIG. 2 and FIG.5, it is understood that light from each pixel x_(i) of TSLM 14illuminates an entire row of partitions 46 of phototransmissive screen24. The transmissive condition of each partition 46 determines theintensity of the light passed therethrough to its underlying hologram44. As seen from FIG. 6, the holograms associated with each column ofpartitions 46 interconnect each pixel x_(i) of TSLM 14 with itscorresponding pixel y_(i) on photodetector 32 and RSLM 36. The intensityat each pixel y_(i) of photodetector 32 and RSLM 36 is thus given by

    Y.sub.i =a.sub.i 1 X.sub.1 +a.sub.i 2 X.sub.2 +a.sub.i 3 X.sub.3 +a.sub.i 4 X.sub.4.

As discussed in the foregoing, photodetector 32 automatically performsthe summation to complete the matrix multiplication ΔX=Y and thereaftercarries out the nonlinear transformation according to G(Y) to obtain Y'.

When the error vector E is applied to RSLM 36, the reflectivity of eachpixel y_(i) therefore is adjusted accordingly to direct each of the fourmodulated and weighted interconnecting beams as probe beams onto PCM 42.The PCM 42 in turn generates amplified return beams which retrace thepath of the probe beams so that when reflected by RSLM 36, the returnbeams are directed back to expose the same partition 46 which passed theinterconnecting beam providing their respective probe beam. When exposedto the return beams, the partitions 46 darken to reduce the intensity ofthe return beams and cause the output vector Y' to approach the desiredpattern vector T.

Previously, it has been stated that the photosensitive screen in theinterconnector of the present invention could comprise a photodichroicmaterial. FIG. 7 shows an alternative embodiment of an interconnector122 together with means 150 for modifying the weights of theinterconnections made by interconnector 122. Interconnector 122comprises a photodichroic screen 124, and a holographic means 126.Except that screen 124 comprises a photodichroic material, screen 124can be constructed in essentially the same way as interconnector 22 ofFIG. 2. It is contemplated that screen 124 comprises a sheet of sodiumfluoride crystals with lithium doping, or a sheet comprising potassiumfluoride crystals. Preferably a doped sodium fluoride sheet comprisesabout 0.175 lithium by weight and measures about 2.5 cm by 2.5 cm on aside. Such a preferred sheet has a thickness of approximately 0.2 mm.

It is contemplated to modify screen 124 by either direct or suppressiveextinction writing. These techniques for writing and readingphotodichroic materials generally, and lithium doped N_(a) F materialsin particular, have been discussed at length in several articles,notably Casasent and Caimi, "Combined Suppressive And Extinction WritingUsing M and M_(A) Centers and Li-doped NaF", Appl. Phys. Lett. (Vol. 29,No. 10, 15 Nov. 1976); Schneider, "Extinction Technique For OpticalStorage Using Anisotropic Color Centers In Alkali Halides" , AppliedPhysics Letters, (Vol. 25, No. 1, July 1, 1974); Casasent and Caimi,"Photodichroic Crystals for Coherent Optical Data Processing", Opticsand Laser Technology, (Apr. 1977); and Casasent and Caimi, "SuppressiveWriting in LithiumDoped NaF", (Phys. Stat. Sol. (a)34.461, 1976) whichare incorporated by reference herein.

As discussed in detail in the above-listed articles, extinction writinguses the anisotropy and dispersion of "M" or color centers in the screen124. An "M"center is defined as two nearest neighbor "F" centers along a<011> direction. An "F" center is defined as an electron trapped at anegative-ion vacancy. Depending upon the alignment of their vacancyaxes, the M centers are described as existing in a "O" state, i.e.state-0, or a re-oriented state referred to as state-1. M centers havean absorption spectrum substantially comprising two bands, an M band atλ_(R) ≃506 nm and an M_(F) band at λ_(W) in the range of 300 to 400 nm.

In direct writing, the entire screen 124 is exposed to λ_(W0) light,having a polarization P_(O). Exposure to the λ_(W0) light initiallyaligns most of the M centers to state-0, that is the λ_(W0) lightbleaches the screen. Thereafter when particular partitions of the screen124 are exposed to λ_(W1) light having a different polarization P₁, theM centers of the exposed portions will re-orient to state-1 to carry outwriting. The screen then can be nondestructively read by exposing it toread light, λ_(R) in the M band.

For interconnector 122, suppressive extinction writing is preferredbecause the same wavelength λ_(R) can be used for both writing andnon-destructive reading of the screen 124. Interconnector is thusdepicted with a source 152 of ultraviolet (UV), λ_(W) light. Accordingto the theory of suppressive writing, the entire screen 124 iscontinuously exposed to UV light in the M_(F) band which thus tends toorient the M centers to state-1. However, in partitions of screen 124which are simultaneously illuminated by λ_(R) light in the M band, there orienting affects of the UV, λ_(W) light are suppressed.Non-destructive reading is performed only with λ_(R) light. Accordingly,modification of the screen 124 is controlled by the λ_(R) light.

Screen 124 is modified by suppressive extinction writing due to thepresence of a polarizor 154 located in front of photodetector 132.According to this technique, spatially modulated λ_(R) light from aninput TSLM 114 is linearly polarized, for example vertically polarized.Polarizer 154 then would be a horizontal polarizor so thatinterconnector 122 is in the optical path between crossed polarizors. Ifthe concentrations of M centers in state-0 and state-1 are equal, thevertical polarization of λ_(R) is unchanged so that all of the light isabsorbed by polarizer 154. If M₀ ≠M₁, then λ_(R) is transmitted tophotodetector 132 at an intensity given by

    I=I.sub.0 [1/4(a.sub.0 -a.sub.1).sup.2 +a.sub.0 a.sub.1 sin.sup.2 (πd(n.sub.v -n.sub.b)λ.sub.R ]

where I₀ is the incident light intensity, a₀ and a₁ are amplitudes ofthe incident light polarized along [011]and [011]respectively, d is thescreen thickness and n_(v) and n_(h) are the refractive indices forvertically and horizontally polarized light respectively. Computer 138receives magnitude intensity signals for each pixel defined forphotodetector 132 following a nonlinear thresholding operation by thephotodetector. Based upon a comparison of the threshold output signalsfrom photodetector 132 with the desired input, computer 138 controlsTSLM 136 and PCM 142 to generate return beams which change therespective state-1 and state-0 M center concentrations and therebymodify the transitivity of the screen 124 to provide an output patternsimilar to the input.

For applications such as recognizing very complex objects or patterns, acomposite network comprising several constituent systems according tothe present invention can be provided. An example of such a compositenetwork 200 is shown in block-diagram form in FIG. 8. Exemplarycomposite network 200 has a first stage 202 of four neural networks210A, 210B, 210C and 210D, and a second stage 204 with one system 210Ewhich receives output from the other four. Each system 210A through 210Ecan be arranged essentially like optical network 10 discussed inconnection with FIG. 2. In operation, each of first stage systems 210A,210B, 210C, and 210D are made to learn to recognize a particular objector pattern which would be a portion of a larger pattern to be recognizedby composite network 200. For instance, first stage network 210A mightbe made to recognize the wing of a plane, system 210B, the forwardportion of the fuselage, system 210C, the tail portion, and system 210D,markings or decals on the body of the plane. When the outputs from thefirst stage systems 210A through 210D are provided to second stagesystem 210E, system 210E would recognize the composite object as aparticular type of aircraft.

It is to be understood that there can be various changes andmodifications to the preferred embodiments of the system of the presentinvention disclosed herein, which changes and/or modifications may bemade by one ordinary skill in the art, but such would still result in asystem well within the scope of the invention as set forth in theclaims.

What is claimed is:
 1. A highly interconnected optical neural networkhaving capability to learn, said network comprising:spatial modulatingmeans responsive to an input signal for modulating light to providemodulated optical signals; detecting means for providing detectionsignals in response to light incident thereon; interconnector means forproviding a distribution of modulated interconnecting light beams tosaid detecting means when said interconnector means is illuminated bymodulated optical signals from said modulating means so as tointerconnect said modulating means and said detecting means according toa predetermined interconnection pattern, said interconnector meansincluding projections means adapted to project a distribution ofinterconnecting beams along paths without changing a parameter of eachof said beams relative to the other interconnecting beams projectedthereby, and interconnection weighting means, provided in said paths soas to be illuminated by said interconnecting beams before said parameterof any of said beams has changed, of controlling said parameter of eachinterconnecting beam relative to the other interconnecting beams toassign an interconnection weight to each said beam and thereby provideweighted, modulated interconnecting beams; and means responsive to saiddetection signals for changing a condition of said interconnectionweighting means to thereby modify said interconnection weights assignedby said weighting means.
 2. A network as claimed in claim 1, whereinsaid interconnection weighting means comprises a photosensitive screenwhich controls the intensity of each interconnecting beam.
 3. A networkas claimed in claim 2, wherein said photosensitive screen comprises aphotochronic material.
 4. A network as claimed in claim 3, wherein saidphotochromic material is a photochromic glass.
 5. A network as claimedin claim 2, wherein said photosensitive screen comprises a photodichroicmaterial
 6. A network as claimed in claim 5, further comprising a sourceof UV light for illuminating said screen.
 7. A network as claimed inclaim 6, further comprising an analyzing polarizer disposed between saidinterconnector means and said detecting means.
 8. A network as claimedin claim 2, wherein said photodichroic material comprises sodiumfluoride doped with about 0.178 percent lithium by weight and saidscreen has a thickness of about 0.2 mm.
 9. A network as claimed in claim7, wherein said photodichroic material comprises potassium fluoride. 10.A network as claimed in claim 2, wherein said projection means comprisesa page-oriented holographic means having an array of spatially-localizedholograms associated in one-to-one correspondence with cell partitionsdefined on said screen.
 11. A network as claimed in claim 2, whereinsaid detecting means comprises photodetecting means for generatingsignals indicative of the intensity of the weighted, modulatedinterconnecting beams and means for nonlinearly transforming suchgenerated signals according to a sigmoidal function to provide detectionsignals.
 12. A network as claimed in claim 2, wherein aid photosensitivescreen comprises regions having optical transmittivities, and whereinsaid means for changing a condition of said interconnection weightingmeans comprises:error signal generation means for generating errorsignals as a function of (a) said detection signals, (b) signalsindicative of a spatial pattern forming a desired output, and (c)modulated optical signals provided by said modulating means and; opticalmeans responsive to said error signals for selecting said regions andvarying the transitivity thereof in response to said error signals. 13.A network as claimed in claim 12, wherein said generation meanscomprises a computer.
 14. A network as claimed in claim 13, wherein saidmeans for selecting said regions and varying said transitivity thereofcomprises:a spatial light modulator disposed so that weighted, modulatedinterconnecting beams provided by said interconnector means are incidenton said spatial light modulator of said varying means in the samepattern that weighted, modulated interconnecting beams are incident onsaid detecting means and, a phase conjugate mirror means placed inoptical communication with said photosensitive screen by said spatiallight modulator of said selecting and varying means.
 15. A network asclaimed in claim 14, wherein said phase conjugate mirror means providesamplified return beams.
 16. A highly interconnected optical processingapparatus for simulating a highly interconnected neural network bycarrying out the matrix operation steps of:(1) AX=Y (2) NL(Y)=Y' (3)E=F(Y', T, X) (4) Z=E² Y (5) Δa_(i) j =ce² _(j) X_(i) a_(i) j,saidapparatus comprising: input means including a spatial light modulatordefining pixels for projecting a pattern of modulated light beamsrepresenting input data in the form of an input vector X with eachmodulated beam representing an element X_(i) of X; a detecting meansincluding a photodetecting means which defines pixels that areassociated with said pixels of said spatial light modulator, and anonlinear transforming means; interconnector means disposed in theoptical path between said spatial light modulator and said detectingmeans for providing a distribution of modulated and weightedinterconnecting light beams to interconnect each pixel of said spatiallight modulator with each pixel of said detecting means when saidinterconnector means is illuminated by a pattern of modulated beams fromsaid spatial light modulator, said interconnector means includingprojection means adapted to transmit modulated light beams from saidspatial light modulator as interconnecting beams without relativelychanging the intensities thereof, and photosensitive screen means forcontrolling the relative intensities of interconnecting beams incidentthereon from said projection means to encode each such beam to have aninterconnection strength according to the element of the matrix A, saidphotodetector detecting the sum of the intensities of the modulatedinterconnecting beams incident on each pixel thereof to provide summedintensity values representative of the result of the matrixmultiplication (b 1) AX=Y where Y is a vector, and said nonlineartransforming means transforming each of the summed intensity valuesaccording to a sigmoidal function to provide detection signals whichrepresent the nonlinear vector transformation (2) NL(Y)=Y' where Y') isa vector; and means responsive to input data representative of thevector X, provided detection signals representative of the vector Y',and data indicative of a transmissive vector T, which vector Trepresents a desired output pattern for modifying a condition of saidphotosensitive screen means to thereby change the interconnectionweights of the modulated interconnecting beams, said means for modifyinga condition of said photosensitive screen including means for generatingan error vector signal representative of the result of (3) E=F(Y', T, X)and optical means responsive to the error vector signal and the vector Yfor generating modifying signals representative of the result of thematrix multiplication (4) Z=E² Y; said photosensitive screen meansresponding to said modifying signals to modify the intensities of theinterconnecting beams according to the result

    Δa.sub.i j =ce.sup.2.sub.j X.sub.i a.sub.i j.


17. A processing apparatus as claimed in claim 16, wherein saidphotosensitive screen means comprises a photochromic glass.
 18. Aprocessing apparatus as claimed in claim 16, wherein said photosensitivescreen means comprises a photodichroic screen.
 19. A processingapparatus as claimed in claim 18, further comprising a UV light sourceadapted to completely expose at least one side of said screen and ananalyzing polarizor disposed in an optical path from said interconnectormeans to said photodetecting means.
 20. A processing apparatus asclaimed in claim 16, wherein said projection means comprises holographicmeans in the form of an array of spatially-localized holograms.
 21. Aprocessing apparatus as claimed in claim 20, wherein said computercalculates the error vector E=F(Y', T, X) according to thealgorithmE=c(Y'-T)^(1/2) where c is the amplification factor of saidphase conjugate mirror.
 22. A processing apparatus as claimed in claim16, wherein said error signal generating means comprises a computer andsaid responsive optical means comprises a phase conjugate mirror placedin optical communication with said photosensitive screen by a reflectivespatial light modulator.
 23. A highly interconnected neural networkhaving capability to learn, said network comprising a plurality of firststage neural network devices which are each adapted to provide anoutput, and a second stage neural network device receiving as inputs,the output of each of said first stage network devices, each of saiddevices includingmeans for receiving optical signals as input; detectingmeans for providing detection signals; interconnector means forproviding a distribution of modulated interconnecting light beams tosaid detecting means when illuminated by optical signals transmittedfrom said input means in order to interconnect said input means and saiddetecting means, said interconnector means comprisingprojection meansadapted to project a distribution of interconnecting beams along pathswithout changing a parameter of the beams relative to each other, andinterconnection weighting means including a photosensitive screenlocated remote from said projection means and in the paths of theinterconnecting beams projected therefrom, for controlling saidparameter of each interconnecting beam relative to the otherinterconnecting beams in order to assign an interconnection weight toeach such beam and thereby provide weighted, modulated interconnectingbeams; and means responsive to detection signals from said detectingmeans for changing a condition of said interconnection weighting meansto thereby modify said interconnection weights.